The present invention relates to a monitoring system and a monitoring method for monitoring around by using cameras each having a fish-eye lens.
A monitoring system using cameras having a fish-eye lens is disclosed in, for example, the Japanese Patent Application Laid-open No. 56-16948. According to a system disclosed in this publication, an image of a wide space is captured by a camera which includes a fish-eye lens having a visual field range of approximately 180xc2x0, and a necessary range is taken out from image data obtained by the camera. Then, the extracted image data is electronically transformed to eliminate distortions of the image due to lens characteristics and two-dimensional image data is generated, and this are displayed in a display unit.
Further, in order to cover a visual field range (monitoring range) of 360xc2x0, there is considered a method in which two monitoring cameras 1 and 3 having fish-eye lenses 2 and 4 respectively are prepared and the cameras 1 and 3 are disposed in such a state that the fish-eye lenses 2 and 4 face mutually opposite directions, as shown in FIG. 9(a).
According to the structure shown in FIG. 9(a), however, the portion between the two fish-eye lenses 2 and 4 becomes entirely a blind area although each of these lenses has a visual field range of approximately 180xc2x0. Since it is not possible to completely adhere the fish-eye lens 2 and the fish-eye lens 4 closely together because of the physical thickness of the fish-eye lenses 2 and 4, the focal length of the lenses, the physical thickness of an image pick-up device (CCD), etc., a band-shaped blind area is formed within an approximately spherical visual field range, as shown in FIG. 9(b). Accordingly, even if the cameras 1 and 3 are disposed in a vertical direction or a horizontal direction, there occurs a gap (a range of which image cannot be picked up) between an image captured by the camera 1 and an image captured by the camera 3.
FIG. 10 shows a case where the fish-eye lenses 2 and 4 shown in FIG. 9(a) are arranged so that the optical axis of each lens is set along a horizontal direction. As shown in FIG. 10, there is a range in which the cameras 1 and 3 cannot monitor around the cameras 1 and 3 because the backside of each of the fish-eye lenses 2 and 4 is a blind area. Accordingly, there is a risk that a suspicious character approaches the cameras 1 and 3 from this area that cannot be monitored without being captured, and destroy the cameras.
FIG. 11 is a diagram for showing a case where the center axis of each of the fish-eye lenses 2 and 4 shown in FIG. 9(a) is set along a vertical direction. In this case, if the fish-eye lenses 2 and 4 are installed at a height higher than the height of a person as shown in FIG. 11(a), the fish-eye lens 2 positioned at the upper side does not play a role. On the other hand, if the fish-eye lenses 2 and 4 are installed at a height near the floor as shown in FIG. 11(b), the fish-eye lens 4 positioned at the lower side does not play a role.
Accordingly, in order to effectively utilize both fish-eye lenses, it is desirable that the fish-eye lenses 2 and 4 are installed within a range of the height of an upper part of the body of a person having an average height as shown in FIG. 11(c). However, in this case, there is a risk that the image of the face of a suspicious character is not captured by any one of the cameras 1 and 3. Further, since it is not possible to capture an image of the total shape of a person in this case, it becomes difficult to specify physical characteristics of the suspicious character based on the images obtained from the cameras 1 and 3.
In order to overcome the above-described difficulties, it is an object of the present invention to provide a monitoring system and a monitoring method that avoid a gap between image pick-up ranges of two cameras by localizing and minimizing the blind area and which monitor around unlike conventional systems and methods.
A monitoring system of the present invention includes: a first camera having a fish-eye lens; a second camera having a fish-eye lens and disposed opposite to the fish-eye lens of the first camera in a direction to face each other; an image data extracting unit for extracting first and second frame data corresponding to a monitoring range from first image data captured by the first camera and second image data captured by the second camera; and an image combining unit for generating combined frame data corresponding to a monitoring range by combining first and second frame data extracted by the image data extracting unit.
According to the above-described structure, an image pick-up range of the first camera and an image pick-up range of the second camera are partially overlapped and only a relatively small area at the backside of each camera becomes the blind area. Frame data of a portion corresponding to the monitoring range are extracted from the image data obtained by the first camera and the second camera, and these frame data are combined by the image combining unit so that combined frame data corresponding to the monitoring range is generated. Even when the monitoring range extends over the image pick-up range of the first camera and the image pick-up range of the second camera, it becomes possible to obtain image data for showing the monitoring range without a gap.
The combined image of the monitoring range can be displayed in a display unit.
When the optical axis of each of the fish-eye lenses of the first and second cameras is disposed to be substantially parallel with a vertical direction, it becomes possible to catch substantially a complete image of the total shape of a person around the first and second camera. It is desirable that the optical axes of the fish-eye lenses of the first and second cameras substantially match together.
Further, the monitoring system may be additionally provided with a transformer for transforming the first and second frame data respectively, extracted by the image data extracting unit, before combining these frame data by the image combining unit. The transformer may be provided with a function for eliminating distortions of an image due to the characteristics of the fish-eye lenses of the first and second cameras.
In order to increase the efficiency of data transfer from the image combining unit to the display unit, it is desirable to provide an image compressing unit for compressing the combined frame data output from the image combining unit and for sending the compressed frame data toward the display unit.
The image combining unit extracts frame data of a non-overlapped range captured by only the first camera from the first transformed frame data, extracts frame data of a non-overlapped range captured by only the second camera from the second transformed frame data, extracts frame data of an overlapped range from at least one of the first and second transformed frame data, and combines the frame data of the two non-overlapped ranges and the frame data of the overlapped range, to thereby generate combined frame data.
Further, the image combining unit can extract frame data of the overlapped range from both the first and second transformed frame data to improve the resolution of the frame data of the overlapped range by interpolating these data. Since the overlapped range corresponds to the area of which resolution becomes low when the data is transformed, it is desirable to improve the resolution by interpolating both image data of the overlapped range. The size of the overlapped range can be easily adjusted by adjusting the distance between the fish-eye lenses.
The monitoring system of the present invention can be installed in any place either indoors or outdoors. In case of indoors, it is desirable to install the monitoring system in a relatively wide room or a passage of a building, such as, for example, each room (or floor) of a store like a department store or a supermarket, a bank, an art museum, a museum, a library, a cinema house, a lobby of a hotel, etc. In the case of outdoors, it is desirable to install the monitoring system around a station, a car parking area, a play ground, a zoological garden, a park, a house garden, etc. Although it is desirable that the monitoring system of the present invention is used for monitoring a presence or absence of a suspicious character within a specific area, the monitoring system of the present invention may also be used for monitoring an animal or a plant.
Further, a monitoring method of the present invention includes: a first step for capturing an image by a first camera having a fish-eye lens and for capturing an image by a second camera having a fish-eye lens and disposed opposite to the fish-eye lens of the first camera; a second step for extracting image data of a range corresponding to a monitoring range from first image data captured by the first camera and second image data captured by the second camera respectively; a third step for transforming the first image data and the second image data corresponding to the ranges extracted in the second step into first transformed frame data and second transformed frame data respectively; a fourth step for generating combined image data corresponding to a monitoring range by combining the first transformed frame data and the second transformed frame data obtained in the third step; and a fifth step for displaying in a display unit an image based on the combined image data generated in the fourth step.
To improve the efficiency of data transfer, it is desirable to include a sixth step for compressing combined image data generated in the fourth step and for transmitting the compressed data toward the display unit.
In the fourth step, it is possible to extract the frame data of a non-overlapped range obtained by only the first camera, the frame data of a non-overlapped range obtained by only the second camera and the frame data of an overlapped range obtained in superposition by the first camera and the second camera respectively, from the first transformed frame data and the second transformed frame data obtained in the third step, and to combine these image data to generate combined image data.